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Abstract 

This article examines the Mathematics, Engineering, Science Achievement 
(MESA) program and investigates its impact on underrepresented student 
populations. MESA was started in California during the 1970s to provide 
pathways to science, technology, engineering, and mathematics careers for 
underrepresented students and represents an exemplar model of informal 
learning environments. Using a mixed-method research design of investigation, 
this exploratory study looks at the relationship between MESA activities and 
underrepresented students’ self-efficacy, interests, and perceptions related to 
engineering. Evidences for this study includes data from focus-group interviews 
conducted and results from quantitative data collected using the Engineering, 
Self-Efficacy, Interests, and Perceptions Survey (ES1PS) instrument. Results 
from this study suggest that participation in MESA’s activities has a positive 
influence on underrepresented students’ self-efficacy, interests, and perceptions 
related to engineering. 

Keywords: Informal learning, underrepresented student populations, mixed 
methods research 

Broadening the participation of underrepresented populations in the science, 
technology, engineering, and mathematics (STEM) fields is a matter of national 
security and has become an emphasis for national policy (Strayhorn, 2015). Yet, 
recent studies have provided evidence that efforts to address these shortages in 
STEM areas have fallen short. In describing patterns of enrollment in STEM 
majors, studies revealed that "less than 15% of undergraduate degrees in 
engineering, math, and physical science were earned by African American, 
Latina/o, or Native American . . . students (NSF, 2013)” (MacPhee, Farro, & 
Canetto, 2013, p. 348). A recent report revealed a troubling trend for 
underrepresented student populations entering into STEM majors. Of all the 
bachelor degrees awarded in 2015, Black students represented only 3% of this 
population, Flispanic students a slightly better 8%, and female students only 
19% of all engineering degrees awarded, numbers that are significantly lower 
than their representation in the general population (National Science Foundation, 
National Center for Science and Engineering Statistics, 2015). Continuing 
efforts have tried to address the proportion of participants in engineering who 
are women and underrepresented minorities, but the demographics of 
engineering enrollments continue to fall significantly short of the goals of 
reflecting the demographics of the overall population (Watson & Froyd, 2007). 
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The result is a STEM field that remains overwhelmingly White, male, and able- 
bodied, leaving the available pool of talented women, minorities, and persons 
with disabilities significantly underrepresented (May & Chubin, 2003). To meet 
this challenge, it is important to identify factors that may help encourage 
underrepresented student populations to choose careers in STEM fields. 

It can be argued that the lack of engineering understanding and a loss of 
interest in science and mathematics is contributing to the lack of 
underrepresented students pursuing engineering careers (Jeffers, Safferman, & 
Safferman, 2004). To effectively address this problem, educators have sought to 
create new and innovative pathways for attracting a talent pool to STEM 
professions that encompasses the diversity evident in the nation’s general 
population (Chubin, May, & Babco, 2005). The National Academy of 
Engineering’s Committee on K-12 Engineering Education released a report that 
detailed the status of engineering in K-12 education (Katehi, Pearson, & Feder, 
2009). In this report, the committee stressed the importance of developing 
curricula with features that appeal to students from underrepresented groups 
(Katehi et ah, 2009). Scientists, engineers, and scholars should not leave the job 
of recruiting underrepresented populations to STEM careers solely to K-12 
teachers of math and science education (Jeffers et ah, 2004). Studies show that 
formal learning environments have traditionally struggled to effectively 
introduce STEM content and STEM professions to underserved student 
populations (Denson, Austin, & Hailey, 2012). 

Currently, there is a lack of empirical research on the efficacy of 
intervention programs to influence underrepresented students (Dyer-Barr, 2014). 
It is important that comprehensive research studies are employed to help 
illuminate the practices that are particularly effective in recruiting 
underrepresented students to STEM careers. One way to address the dearth of 
literature on best practices for recruiting underrepresented students to STEM 
careers is to investigate the practices of informal learning environments, 
particularly those that have been effective in recruiting underrepresented 
students to STEM careers. 

Informal Learning Environments 

Informal learning environments may provide the milieu needed for 
introducing STEM content to all students, but even more importantly, they may 
provide a pathway to STEM careers for underrepresented students. It is 
estimated that students spend 86.7% of their time outside of a classroom 
(Gerber, Cavallo, & Edmund, 2001). This helps illustrate the importance of 
informal learning environments and the opportunities that they may provide for 
the teaching and learning of STEM content. Martin (2004) notes that informal 
learning environments have been an integral part of education for years and will 
be critical for transforming the teaching of STEM content in the 21st century. 
Although the merits of informal learning environments are duly noted, research 
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in this area is sparse and undecided on how these experiences benefit students 
(Gerber et al., 2001). Beyond anecdotal reporting on informal learning 
environment experiences, there is little research detailing specific activities and 
their effect on students. This highlights the need to investigate informal learning 
environments that effectively teach STEM-based concepts to students. Although 
there are aspects of the program that are conducted during school hours, MESA 
formally functions as an afterschool program complementing the work of formal 
STEM curricula. Inferential studies into the ways that informal learning 
environments are able to impact underrepresented student populations are of 
particular importance. The results of investigations that explicate how successful 
informal learning environments impact underrepresented students will provide 
insight into how the United States can attract diverse populations to STEM 
fields. 

Chubin, May, and Babco (2005) produced a review of engineering-based 
informal learning environments and concluded that effective engineering-based 
informal learning environment “must (1) promote awareness of the engineering 
profession, (2) provide academic enrichment, (3) have trained and competent 
instructors, and (4) be supported by the educational system of the student 
participants” (p. 79). Categorically, informal learning environments fall into 
three settings: (1) “everyday experiences,” (2) “designed settings,” and (3) 
“programmed settings” (Kotys-Schwartz, Besterfield-Sacre, & Shuman, 2011, p. 
1). Program settings are characterized by “structures that emulate [or 
complement] formal school settings—planned curriculum, facilitators or 
mentors (taking a teaching role), and a group of students who continuously 
participate in the program [(National Academy of Sciences, 2009)]” (Kotys- 
Schwartz et al., 2011, p. 2). The learning environment featured in this study, the 
Mathematics, Engineering, Science Achievement (MESA) program has been 
identified as an effective informal learning environment and is categorically 
identified as a programmed setting (Mathematics, Engineering, Science 
Achievement [MESA], 2017a). Research has shown that students who 
participate in the MESA program “outperform California public high school 
students overall in the following categories: completion of advance mathematics 
and physics courses, grades and performance on college entrance exams 
[(Building Science and Engineering Talent, 2004)]” (Kotys-Schwartz et al., 

2011, p. 2-3). Due to MESA’s success as an informal learning environment and 
its unmatched ability to recruit and retain underrepresented student populations 
to STEM careers (MESA, 2017a), researchers for this study were interested in 
examining the aspects of MESA that appealed to their underrepresented student 
populations. 

This article reports on the results of an investigation into the impact of the 
MESA program on underrepresented student populations. Using a sequential, 
exploratory, mixed-method research design, this article adds to the literature 
focused on underrepresented student populations and informal learning 
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environments. This article will first provide the reader with a brief history of the 
MESA organization followed by the research design framing this study. The 
article will follow with results from focus-group interviews conducted with 
MESA participants, which provided eight intriguing themes from the MESA 
organization and helped informed the design of the Engineering, Self-Efficacy, 
Interests, and Perceptions Survey (ES1PS) instrument used in this study. Finally, 
this article will provide results and conclusions from quantitative data collected 
from over 400 student participants using the ES1PS instrument. 

The MESA Organization 

The first MESA program was founded in 1970 at Oakland Technical High 
School in Oakland, California with a membership of 25 students. MESA’s goal 
was “to develop academic and leadership skills, raise educational expectations, 
and instill confidence in California’s students’’ from groups that were 
"historically underrepresented in engineering, physical science, or other math- 
based fields in order to increase the number of African American, Latino 
American and American Indian graduates from a four-year university” (MESA, 
2017, para. 1). The MESA effort was supported by the California Public School 
System, the state Community College System, and the California College 
System. “There may be other established programs, or programs under 
development, designed to increase Latino academic achievement in mathematics 
and science, but none has the longevity, organizational structure, network, and 
academic rigor as does MESA” (Haro, 2004, pp. 218-219). MESA has been 
able to achieve these goals despite declining federal and state support. 

MESA supports educationally disadvantaged students and minority students 
in middle schools and high schools by providing pathways to help them succeed 
in science, mathematics and engineering (Kane, Beals, Valeau, & Johnson, 
2004). MESA’s goals are to: (1) "increase the number of engineers, scientists, 
mathematicians, and related professionals at technical and management levels, 
and (2) serve as a driving force in encouraging minorities and females in 
achieving success in these fields” (Maryland MESA, 2012). MESA programs 
are based on a common co-curricular academic enrichment model that includes 
“academic planning, community service, family involvement, academic 
enrichment, hands-on engineering activities, career advising, field trips, 
competitions and workshops”(MESA USA, 2011). MESA programs represent 
an innovative way of linking a co-curricular learning environment to 
mathematics, engineering, and science programs within the formal public-school 
setting to enhance the STEM education of students. 

Over the past 40 years, the California MESA program has become a model 
for MESA-USA, a partnership that now involves MESA programs from nine 
states that are joined together to support disadvantaged and underrepresented 
students to improve their academic achievement in math, science, and 
engineering. MESA-USA members are active in Arizona, California, Colorado, 
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Maryland, New Mexico, Oregon, Utah, Washington, and Pennsylvania. 
Additional information about the history and status of MESA are available on 
their website: https://mesa.ucop.edu/about-us/. 

MESA “has demonstrated through statistics from their California statewide 
office that MESA students outperform California public school students overall 
in the following categories: completion of advanced mathematics and physics 
courses, grades and performance on college entrance exams [(Building Science 
and Engineering Talent, 2004)]” (Kotys-Schwartz et al., 2011, pp. 2-3). A 
review of evaluation reports from after-school science, technology, engineering, 
and mathematics (STEM) programs, both co-curricular and extracurricular, by 
the Afterschool Alliance found “that attending high-quality STEM afterschool 
programs yields STEM-specific benefits that can be organized under three broad 
categories: improved attitudes toward STEM fields and careers; increased 
STEM knowledge and skills; and higher likelihood of graduation and pursuing a 
STEM career” (Afterschool Alliance, 2011, p. 2). Further evidence of the 
program’s impact is that California MESA received the Presidential Award for 
Excellence in Science, Mathematics, and Engineering Mentoring in 2000, an 
award administered by the National Science Foundation on behalf of the 
President. 

The success of individual MESA programs has been well documented, 
including a recent external evaluation conducted by John Flopkins University 
with the Maryland MESA program (Corcoran, Eisinger, Reilly, & Ross, 2014). 
This study looked at MESA’s influence on students interest, but it was limited to 
one state and included just 77 participants (Corcoran et al., 2014), making the 
results hard to generalize. There is still a need for empirical research that 
identifies “appropriate content for informal learning models [or environments] 
or . . . assessfes] the degree to which these informal experiences impact 
students” (Kotys-Schwartz et al., 2011, p. 1), particularly across programs in 
different states. In response to this need, this article will present qualitative and 
quantitative data to illustrate the ways in which MESA is able to influence 
students’ self-efficacy, interest, and perceptions of engineering. Furthermore, 
relationships among students’ interest, perceptions, and self-efficacy will be 
explored, and qualitative data will be presented on the benefits of MESA for 
underrepresented students. This study was designed to examine students’ 
participation and involvement in five activities that are common among MESA 
programs: field trips, guest lecturers, design competitions, hands-on activities, 
and career and academic advisement. 
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Research Design 

Methodology 

This study utilized a mixed-method research design. The purpose of this 
study was to examine the MESA program and understand features of the 
program that appeal to underrepresented groups. This work complements the 
work of Tierney and Farmer (2002), by identifying student-oriented activities 
within the MESA program that have an influence on underrepresented students’ 
engineering self-efficacy, interest in engineering and perceptions of engineering. 
In addition, focus-group interviews were conducted in an effort to unpack 
activity variables within the MESA organization and understand the benefits of 
the program for underrepresented students. The study was conducted in four 
MESA-USA states: California, Maryland, Washington, and Utah. This study 
used qualitative and quantitative measures to answer the research questions. The 
first research question was addressed in the qualitative portion of the study. 

1. What are the benefits of participating in MESA for underrepresented 
student populations? 

The second, third, and fourth research questions were addressed in the 
quantitative portion of the study. 

2. What influences do MESA activities have on students’ engineering 
self-efficacy? 

3. What influences do MESA activities have on students’ interest in 
engineering? 

4. How are the students’ perceptions of engineering influenced by their 
participation in MESA activities? 

This study examined student-oriented activities which can be categorized 
into five distinct groups: (a) field trips, (b) guest lecturers, (c) design 
competitions, (d) hands-on activities, and (e) student advisement. These five 
MESA activities represented the independent variables for this study. The 
dependent variables for this study included students’ self-efficacy, interest, and 
perceptions related to engineering. The study also examined the influence of the 
MESA program on outcome factors. 

Mixed-method research designs are particularly advantageous when seeking 
to confirm and cross-validate findings within a single study (Creswell, 2009). 
This study employed an exploratory design of investigation. 

Exploratory designs begin with a primary qualitative phase, then the 
findings are validated or otherwise informed by quantitative results. This 
approach is usually employed to develop a standardized (quantitative) 
instrument in a relatively unstudied area. The qualitative phase identifies 
important factors, while the quantitative phase applies them to a larger 
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and/or more diverse sample (Creswell and Piano Clark, 2007). (Borrego, 

Douglas, & Amenlink, 2009, p. 59) 

In this study, focus-group interviews were used to help identify important 
features of MESA, which were applied to a larger sample during the quantitative 
phase. In this study, the qualitative results helped identify features within the 
MESA program that appealed to underrepresented student populations. In 
addition, the focus-group results informed instrument development of the ES1PS 
instrument. 

Theoretical Framework 

The theoretical framework that guided this study was social cognitive 
theory’ (SCT), which holds that knowledge acquisition is directly related to 
observing others within their context of social interactions, experiences, and 
outside media influences (Bandura, 1988). This framework proposes a 
relationship between outcome expectations and other behavioral factors such as 
self-efficacy and interest. SCT is based upon the assumption that human ability 
is a dynamic attribute and that competence in complex tasks requires both well- 
developed skills and a strong sense of efficacy to deploy one’s resources 
effectively. Social cognitive career theory (SCCT) provides a base for exploring 
the interaction among personal, environmental, and behavioral influences in 
career development (Lent, Brown, & Hackett, 1994). This framework is 
appropriate for this study because of SCT’s emphasis on the role that self- 
efficacy, beliefs, outcome expectations, and goals play in career selection. 

Self-efficacy. The first construct to be explored in this study is self-efficacy, 
as defined by Bandura (1977, 1986), which refers to the beliefs about one’s 
ability to execute a given task or behavior in order to attain designated 
performance. Research has provided evidence that the lack of participation of 
minorities in STEM careers is due in part to low self-efficacy in science and 
mathematics. Self-efficacy has been found to be a powerful contributor to the 
decision to pursue a career in STEM and a major predictor of success in STEM 
courses (Zeldin, Britner, & Parajes, 2008). Although studies have examined self- 
efficacy as it related to STEM fields, few have focused specifically on 
engineering (Lent et al., 1994). However, there is evidence that self-efficacy 
regarding scientific-technical tasks is predictive of student interest (Brown, 

Lent, & Larkin, 1989) and academic performance (Hackett, Betz, Casas, & 
Rocha-Singh, 1992) in STEM fields. Bandura (2006) states that "there is no all¬ 
purpose measure of perceived self-efficacy” (p. 307). Sherer et al. (1982) noted 
that self-efficacy has been primarily thought of as a task-specific belief. Thus, in 
order to measure engineering self-efficacy, a scale must be created specifically 
related to the engineering domain. "Self-efficacy scales must be tailored to 
activity domains and assess the multifaceted ways in which efficacy beliefs 
operate within the selected activity domain” (Bandura, 2006, p. 310). Although 
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some researchers have attempted to create an accurate measure of “general self- 
efficacy,” arguments still persist about the scales validity as a true measure 
(Chen, Gully, & Eden, 2001). Sherer et al. (1982) assert that when dealing with 
specific behaviors, more direct behavioral measures will increase the accuracy 
of the measurement. 

Interest in engineering. The second construct in this study is interest. If 
one seeks to account for the low numbers of underrepresented students in STEM 
careers (e.g., Babco, 2001), one need only look at the trend of tracking and the 
placement of minority students in lower academic tracks which has negatively 
impacted student interest in the sciences (Museus, Palmer, Davis, & Maramba, 
2011). Multiple studies describe the importance of interest and its relationship to 
self-efficacy (e.g., Fouad & Smith, 1996; Hutchinson, Follman, Sumpter, & 
Bodner, 2006; Wender, 2004). Bandura (1986) suggested that perceived efficacy 
in people fostered the growth of intrinsic interest, which would remain 
consistent as long as those interests engaged their personal feelings and offered 
satisfaction. The decades old trend of placing minorities in lower academic 
tracks does not foster intrinsic interest and may contribute to shortages of 
minority representation in fields such as science and mathematics (Babco, 2001; 
Boyer, 1983). A lack of interest in learning science and engineering may come 
about if one does not see science or engineering as a viable career option. 
Researchers in science education have asserted that one reason students from 
low-income communities are not interested in science is that there is “a 
‘disconnect’ between school and home/community life” (Basil & Calabrese 
Barton, 2007, p. 467). Currently, research offers few solutions on how to sustain 
these students’ interest. However, Basil and Calabrese Barton (2007) found a 
“strong connection between a sustained interest in science and science learning 
environments in which students were able to cultivate relationships with people 
and in ways that reflected their values of relationships and community” (p. 483). 

Carlone and Johnson (2007) found that interest in science or science-related 
fields had less to do with the subject of science than with the effect that their 
scientific competence would have on the world. The participants in their study 
were interested in humanitarian work such as health care—efforts that could 
change the world in a positive way. Interests, along with self-efficacy and 
outcome expectations, predict intentions, which in turn lead to choice behaviors 
including those about careers (Lent et al., 1994; Waller, 2006). Waller (2006) 
also found that African American students’ “math self-efficacy and outcome 
expectations predicted math interest” (p. 543). Brown et al. (1989) showed that 
even if there is strong interest in a pursuit, if another option is viewed as more 
attainable that will be the one to which students will strive. In addition to these 
findings, Fouad and Smith (1996) found that self-efficacy was a large influence 
on students’ interest. Math and science self-efficacy are included among the 
factors that impact students’ interest in engineering. 
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Perceptions of engineering. The final construct to be explored is students’ 
perceptions about engineering. A student’s perception of an occupation along 
with their self-efficacy in skills associated with that occupation greatly influence 
the likelihood that the student will pursue the occupation (Bandura, Barbaranelli, 
Caprara, & Pastorelli, 2001). For some individuals, their perceived efficacy 
rather than their actual achievement is a key determinant of their perceived 
occupational self-efficacy and preferred choice of work. In a study of African 
American females, Carlton Parsons (1997) found that 11 of the 20 interviewees 
imagined a scientist as an unattractive, nerdy, White male. Their image 
described the male as having a secondary social life with a perfect family. The 
image that they described did not represent what most African American 
students see on a daily basis. In fact, negative attitudes toward engineering and 
"less positive perceptions of the work engineers do” have been reported as key 
factors in high attrition rates for aspiring engineering students (Besterfield- 
Sacre, Atmn, & Shuman, 1997; as cited in Hirsch, Gibbons, Kimmel, Rockland, 
& Bloom, 2003, p. F2A-7). Changing the public’s perception of engineering was 
a major focus of a study reported in the National Academy of Engineering’s 
(2008) report Changing the Conversation: Messages for Improving Public 
Understanding of Engineering. These factors highlight the need to address the 
negative perceptions that underrepresented students have of sciences. 

Qualitative Study: Focus-Group Study 

Purpose 

Focus-group interviews were conducted with two goals in mind. First, the 
researchers were interested in understanding the nuances of the MESA 
organization by unpacking the activity variables of the informal learning 
environment. Second, findings from the focus-group interviews would inform 
instrument development for the quantitative phase of the study. The purpose of 
the focus-group interviews included determining the benefits of participating in 
MESA’s informal learning environment for underrepresented students. 

Methodology 

The research team used a focus-group protocol to guide the interview 
sessions. “Focus groups are used to gather opinions” (Krueger & Casey, 2009, p. 
2). They consist of a series of interviews, conducted with five to 10 participants, 
wherein the researcher attempts to gain a certain perspective from a particular 
group (Krueger & Casey, 2009). Focus-group interviews are well suited for 
qualitative studies including grounded theory (Webb & Kevern, 2001). 

Members of the group are there for member checking, expounding upon 
participant responses, and adding clarity to group responses. 

Focus group interviews typically have five characteristics or features. These 

characteristics relate to the ingredients of a focus group: (1) people, who (2) 
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possess certain characteristics, (3) provide qualitative data (4) in a focused 

discussion (5) to help understand the topic of interest. (Krueger & Casey, 

2009, p. 6) 

In order to ascertain a perspective that was reflective of the MESA program, it 
was important to establish a “consensus” among group members. For the 
purpose of this study, researchers felt that focus-group interviews were 
appropriate. The participants for this study were all members of MESA who 
provided qualitative data during a focused discussion in an effort to inform the 
researchers as to the aspects of MESA that were particularly beneficial to their 
experience. 

Focus-group interviews are particularly beneficial when seeking consensus: 
Interactions among participants enhance data quality because participants 
provide checks and balances on each other’s statements (Patton, 2002). A semi- 
structured interview technique was employed to collect data. During the focus- 
group interviews, the interviewer was allowed to digress and probe the students 
for richer descriptions of activities before returning back to the interview guide 
to maintain the integrity of the interview process (Krueger & Casey, 2009). 

Participant Selection 

Participants were selected for this study using purposeful sampling. 
Purposeful sampling is an effective strategy of sampling that allows for the 
collection of "information-rich” data (Glesne, 2006; Patton, 2002). Advisors for 
each MESA chapter participating in the study selected participants for the focus 
groups based on student attendance, achievement, and overall participation in 
the MESA program. Researchers for this study used a purposeful sampling 
technique in order to secure participants who could provide insight into the 
aspects of MESA that were beneficial and understand what students are gaining 
by their participation. Using a purposeful sample of successful MESA programs, 
researchers were keen in selecting settings and participants who could help 
illustrate characteristics of the MESA program that led to student recruitment 
and retention. It is important to note that this focus-group study was not done in 
an effort to evaluate the effectiveness of MESA; instead, researchers were 
investigating the aspects of the program that helped recruit and retain students. 
Participants were provided with food and refreshments as remuneration for their 
participation. A total of 28 MESA students from five different schools in the 
California area participated in the five focus-group interviews. Due to 
convenience, time constraints, and logistical challenges researchers limited their 
focus-group populations to schools in California. As an example, over a period 
of 1 week, researchers rented a car and travelled to six different schools in 
California to collect the data. The student distribution is as follows: Site 1 
provided seven participants, Site 2 provided five participants, Site 3 provided six 
participants, Site 4 provided five participants, and Site 5 provided five 
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participants. In total, there were 19 females and 11 males for a total of 28 
student participants. For the duration of this article, participants involved in the 
qualitative phase will be referred to as focus-group participants in order to 
distinguish them from the larger student population. 

Data Gathering 

Two researchers were responsible for conducting focus-group interviews. 
For each interview, one researcher served as a facilitator, and the other 
researcher served as a note taker. Because both researchers were well versed in 
conducting qualitative research, they alternated roles as facilitator and note 
taker. The focus-group interviews were audio recorded, and notes were taken to 
ensure that data could be crosschecked with the audio recordings. 

The interviews took approximately 1 hour to complete for each focus group. 
The facilitators posed two open-ended questions. 

1. Can you think of one of the best times you have had in MESA? 

2. What do you think you are gaining by participating in MESA? 

After the first question was asked, the facilitator asked additional probing 
questions for the purpose of clarification and confirmation. This allowed the 
participants to answer a multitude of questions with minimal probing from the 
facilitator. After a number of supplementary questions, the second main question 
was then posed as a concluding question. Again the process was repeated with 
the facilitator listening carefully to answers and asking additional or follow-up 
questions from answers given. The themes formed are the result of four recorded 
interviews and notes taken from a fifth interview. Technical difficulties 
prevented transcription of the fifth focus-group recordings. 

Analysis 

To build towards a theory of impact and influence relative to MESA 
activities and underrepresented students, researchers employed a grounded- 
theory approach to analyze the focus-group results. Grounded theory is an 
inductive, comparative, iterative method that is used primarily as a method of 
data analysis. This strategy is useful when striving to render a conceptual 
understanding from the data (Charmaz, 2001; Dey, 2004). The grounded-theory 
approach yields themes that are formed from the grouping of codes according to 
conceptual categories that reflect commonalities among coded data (Glaser & 
Strauss, 1967). In this study, researchers examined emergent themes formed 
from the focus-group participants’ responses. This was accomplished by looking 
at the transcribed recordings and notes that were taken during each interview 
session. Individual researchers reviewed collected responses and gradually went 
from coding to categories and eventually theory building, which led to the 
development of activity components (Harry, Sturges, & Klingner, 2005). Using 
the research question as a guiding framework, frequency counts were used to 
determine themes that were recurrent in order to identify emergent themes. 
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Stemler (2001) contends that frequency counts should be used to determine 
content of particular interest. In qualitative research, “a summative content 
analysis involves counting and comparisons, usually of keywords or content, 
followed by the interpretation of the underlying context” (Hsieh & Shannon, 
2005, p. 1277). Emergent themes were verified only once consensus was 
reached amongst the researchers regarding these themes. 

In providing a rich, thick description of the emergent themes, quotes were 
used to help paint a picture and to provide “good data” (Morse & Richards, 
2002). Based on the procedures described, readers can have confidence that the 
conclusions and themes that emerged from the study are in fact a reflection of 
the participant’s responses. To establish consensus, researchers met with the 
principal investigator for the study to discuss emergent themes from the 
analysis. Themes included in the results were a result of frequency counts to 
determine recurrence. After recurrent themes were identified independently by 
each researcher, researchers provided evidence in the form of student quotes to 
the Principal Investigator (PI) for this study in order to move into an emergent 
theme. The PI was not involved in collecting or coding the data and was thus 
able to provide an unbiased decision about whether evidence was sufficient to 
justify inclusion as an emergent theme. After themes were agreed upon and 
justified to a third party, headers were developed to easily identify the 
characteristics of the prescribed theme. 

Results and Discussion 

The results of the grounded-theory approach to analyzing the focus-group 
responses produced eight disparate themes that spoke to the benefit of the 
MESA program for underrepresented students: (a) informal mentoring, (b) 
makes learning fun, (c) time management (d) application of math and science, 

(e) feelings of accomplishment, (f) builds confidence, (g) camaraderie, and (h) 
exposure to new opportunities. 

Informal mentoring. A surprising theme emerged from the analysis of data 
from the focus-group interviews. Participants talked more about their roles as 
mentors in informal mentoring settings, as opposed to the informal mentoring 
that they received from MESA teachers and advisors. Participants spoke about 
mentoring not only their fellow underclassmen but also volunteering with local 
middle and elementary school. 

Instead of doing the competition and competing, we get to volunteer—we 
get to help with the um middle schools and um help them make their 
projects and give them advice. (Group 1) 

We tutor elementary schools too, so there’s a lot of elementary schools 
around. (Group 3) 
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Informal mentoring from the MESA advisors and teachers was also mentioned: 

This is a club that like wants people, us, all of us, to succeed in life going 
into college, succeeding in that, all the advisors, all the teachers, just want 
to see you achieve, to your best quality. So they’re going to help you out 
and to be the best you can be in succeeding. (Group 2) 

Makes learning fun. Participants seemed to agree that making learning fun 
was a key part of MESA’s success. They not only spoke of the MESA 
experiences in reference to learning but also voiced the importance of MESA 
experiences in changing their perceptions of STEM fields. 

It is actually really fun, you don’t fall asleep. Um yeah, you don’t fall 
asleep. It’s amazing. (Group 1) 

If we didn’t have the fundamental of math—I mean fun, in between there— 
it would be really boring. (Group 2) 

That’s something that MESA shows you at hand. You actually see people— 
actually see engineers and they’re just out there doing their thing, and 
they’re just having fun and they’re enjoying it. (Group 3) 

Time management. Organization and time management emerged as a 
prevalent theme among the focus groups. When speaking about the benefits of 
MESA, a participant spoke about the impact of the program stating 

Like MESA, like kind of helped me like 1 used to be something like get on 
time, and something like that do some other stuff with MESA and taught 
me that 1 should be doing stuff earlier than doing it at the last second. 

(Group 4) 

The competitions also aided in developing time management skills: 

You learn that time is of the essence because we’re there working, and then 
once we get to Saturday academies, or regionals, everything has to be on 
schedule, or we’re running late, you have to turn in project at the certain 
time, so you’re running. (Group 2) 

MESA advisors helped participants with the organization necessary for 
application to college programs: 

(MESA helps) when, there’s so many deadlines and applications you have 
to turn in as a senior for college. (Group 1) 
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Application of math and science. The focus-group participants expressed 
an understanding of the importance of having opportunities to apply the math 
and science learned in formal learning environments. One participant explained 
the integration of the formal and informal learning environments: 

So as I would do MESA, 1 would get more encouraged and be wait, this is 
what 1 was learning in class. So where I would learn something in class, I 
would use it in MESA, and when I would learn something in MESA 1 
would use it back in my class. You’re realizing that this isn’t just something 
you’re doing for pointless reasons, but you’re doing something with it. 
(Group 2) 

Several participants commented on learning the importance of math and science 
and also the opportunity for transference of knowledge. 

Well the best experience I’ve had in MESA has been just overall learning 
the value and importance of math and science. Because we—we put math 
and science into like—into all these projects we do. (Group 3) 

And MESA really brings out—really tells you like—it really gives you an 
experience of what it’s used for. Like here we’re doing physics in class, and 
I’m like what am I going to use this for? You know, how does this apply to 
me? But then once you do the windmill or something like that.... (Group 
1 ) 

Feelings of accomplishment. MESA provided opportunities for 
participants to achieve outcomes that seemed to be key components in the 
program. Participants voiced feelings about competing and winning 
competitions. 

One thing you get is just this immense sense of accomplishment, that you 
did something and it’s not [sic] something that you just can’t fabricate. 
(Group 2) 

I get an award, 1 get this medal on to show that I put that much effort into it. 
And that’s something that MESA does for us. Well to me, it makes me feel 
accomplished like I actually did something, that I put my work into, and I 
got something out of it. (Group 3) 

We were doing team math, and when we won first place I was, like yes. So 
it was a good time for me... (Group 4) 
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Builds confidence. Participants commented on gaining and building 
confidence from winning competitions and seeing others like them succeed. 

I think I’ve gained a lot of confidence in myself from MESA, because you 
do a project, or you give a speech, or you take a math test and you kind of 
think, 1 don’t know 1 kind of did okay on that, (Group 4) 

Well now that I’ve done it since 7th grade, it’s been easier. Like I’m more 
calm 1 know what I’m doing it’s just as I go through it 1 just learned from 
7th grade don’t be as nervous, just do the best, and just have a little mental 
power that, you know, I can do it and I can. (Group 2) 

Like it make a difference for me because like all my dad, um a lot of his 
friends from college became engineers, so it’s kind of cool because you 
may of his friends that I’ve met have been male engineers, and like I go to 
their companies and it’s all like guys working there. So it was kind of cool 
because like not only was she an engineer, but she was like in charge of 
many projects. And so like it showed like how it didn’t matter so anybody 
could do—be in charge. (Group 1) 

Camaraderie. Although the participants spoke often about participation 
and placing in various competitions, these activities were often times not what 
kept them coming back. One of the prevalent themes that we heard was the 
camaraderie formed by working on projects and visiting different schools while 
participating in different events. Below are a couple of examples from different 
groups expressing such thoughts. 

Like if you’re at prelims you just kind of cheer for your school if they win, 
but when you go to regionals, if your center wins, then you’re cheering for 
them. But it’s not like that serious, like at competitions like you want to win 
of course, that’s what you’re doing it for but like everyone is kind of you 
know relaxed, and everything like everyone talks to everyone. It’s not like, 
you know, you don’t talk to them because they’re your competitor or 
whatever. It’s kind of like oh, you’re here too, how did you do this year? 
(Group 1) 

And when we’re doing the trebuchet, we spent countless hours. We would 
go to our advisors house, stay there from like eight in the morning, and it 
would be eight at night. And we’d be trying to build it. It would be all the 
groups and we help each other. (Group 2) 

we interact with other schools, and we’re—and well, you get to meet new 
people when you’re doing the same project as they are, and they get to give 
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you like what Martha said, and everybody else they get to give you hints on 
what to do on the project, and then besides that, even though you’re 
competing against them, you make new friends that will help you. (Group 
2 ) 

The relationships formed among the groups were paramount to why a number of 
students stay in MESA: 

Well, I stayed with people I didn’t really know during MESA that year that 
well, because they were juniors and 1 was just a sophomore. It was kind of 
fun 1 stayed with them, got to know each other better, got closer for this 
year. So that was really fun to like, you know, all hang out there together. 
(Group 1) 

Exposure to new opportunities. A number of the participants came from 
backgrounds that do not afford them opportunities to visit college campuses or 
to work on projects outside of their classrooms. MESA provided a bridge to 
those participants, which did not go unnoticed or unappreciated. 

MESA, it gives you so many opportunities, that a person like me, would 
never have had. Like my parents were always—like they complain about 
the hours I put in for like my projects, you know, but they’re like oh, you 
should—you should do that because it gives you the opportunity—like 
gives you an opportunity to like see things that we’ll never—that you’ll 
never get to see with us, you know. Because my parents they’re not really, 
like um—they don’t know any English so they can’t go anywhere, so they 
never take me anywhere and they’re just like yeah, so you should just like 
do your best. And join things that would allow you to see others things, you 
know, give you opportunities. And MESA really does that. (Group 1) 

And like one thing is like with engineering and stuff, that it—there’s not a 
club on campus that would allow you to explore that option. There’s some 
for writing and reading, the obvious subjects, but sometimes like 
engineering is kind of like pushed back because it’s math and science, the 
two most unpopular subjects at a school. And then on top of that, you’re 
asked to do a lot of different projects. And without MESA not a lot of 
teachers would be willing to have just the fun option of trying this. (Group 
2 ) 

Just this last weekend they took some of us juniors to Chico—Chico 
University and this took us Sunday night and we slept over there at an 
apartment that these girls share. And so it’s not just the fact that you’re 
there, but you get this feeling like you belong. And it’s um—you’re part of 
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the college. And you get to—you get to experience that even before you go 
to college. And it was really nice we were just—and then we got to visit the 
dorms we got to visit around school. We saw students, ex-MESA students 
from this school and they’re really happy they say their classes are super 
hard, but they’re loving it. And it’s just really nice. And just MESA we’re 
just like—we’re really united when it comes to. (Group 3) 

Quantitative Study 

The final phase of this mixed-method investigation included the quantitative 
portion of the data collection, which included over 700 students responding to 
the survey. The participants came from 22 high schools in the states of 
California, Maryland, Utah, and Washington. Responses selected for analyses 
were from individuals who had participated in MESA during the 2012-2013 and 
2013-2014 school years. In addition to logistical challenges, communication 
proved to be a challenge for many states that had agreed to participate in the 
study. Due to these challenges, data collection extended to over a 2-year period 
in order to include a diverse set of underrepresented student groups. Below the 
researchers describe the selection criteria for the participating schools. 

For prospective schools to be considered for this study they had to adhere to 
the following criteria: 

1. The high school must have an established MESA program that 
provided the five student-oriented activities throughout the school year. 

2. Students selected will represent underrepresented populations in 
engineering. 

3. Underrepresentation is defined as having a representation in a particular 
field that is substantially lower than the representation in the general 
population. 

4. Student participants will have at least one academic year of experience 
in the MESA high school program. 

Data Collection 

After school sites indicated interest in participating in the study, a message 
describing the study and thanking them for their help was sent to the respective 
site. The message described the purpose of the study and outlined instructions 
for the administration of the survey. The advisers were asked to have their 
students complete the survey during a MESA meeting in a room that provided 
computer and Internet access. A letter of information (LOI) was included with 
the request that it be sent home to parents or guardians. Some teachers needed 
hard copies of the LOI, which were sent to them via mail, and others were 
willing to print copies from an e-mailed PDF file of the LOI. Advisers were 
informed that if a parent or guardian did not want their child to participate in the 
study, the parent or guardian should sign and return the form to the adviser. In 
such a case, the adviser did not allow the student to take the survey. The MESA 
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programs at collaborating schools received a stipend for participating in the 
study. Each adviser was given an individual SurveyMonkey® link to enable 
participants to complete their individual survey responses and submit their 
responses electronically to the project office. An exception was made for one 
school because the adviser requested paper copies to overcome the lack of 
student access to computers. The responses from that particular school were 
entered into the database by a project staff member. 

Instrument. Data was collected using the ESIPS instrument. The instrument 
included three subscales for each construct: the self-efficacy subscale contained 
11 items, the perceptions subscale contained 12 items, and the interests subscale 
contained 14 items. There was also an outcomes section that was developed as a 
result of the qualitative work, which contained 39 items. The ESIPS instrument 
is a valid and reliable instrument that is the original work of the authors 
(Denson, Austin, & Hailey, 2014). Cronbach’s alpha was presented as measure 
of internal consistency or reliability for use with the psychometric measure 
(Schmitt, 1996). Results of the reliability test provided evidence that the results 
reported are valid and reliable. Results of the reliability estimates revealed very 
satisfactory scores for each of the subscales: .93 for the self-efficacy subscale, 

.85 for the perceptions subscale, .90 for the interest subscale, and .96 for the 
newly added outcomes subscale. The overall Cronbach Alpha for the survey 
instrument was .96. 

Data collection for the quantitative phase of this study was conducted at two 
different intervals with over 700 students responding to the survey. Participants 
came from 22 high schools in the states of California, Maryland, Utah, and 
Washington. Responses selected for analyses were from those individuals who 
had participated in MESA during the 2012-2013 school year. The responses 
from California, Washington, and most of Maryland schools included in this 
report were obtained during the 2013-2014 school year. The responses from 
Utah schools and several Maryland schools were obtained at the end of the 
2012-2013 school year. Many individuals who started the survey failed to 
complete the instrument but scrolled quickly through the items and exited the 
survey without providing any useful data. The database of returns was scanned 
manually, and partial results from respondents who had not completed at least 
one subscale of the survey were not included in the analysis. After this 
screening, the final sample for the study was composed of 484 students. 

Data from Tabulations of Survey Results 

This section begins with demographic data to orient the reader to the 
characteristics of the sample of respondents. The data provides information 
about the genders of the respondents and their ethnicities. 

Gender. When asked to indicate their gender, 211 respondents indicated 
that they were male (43.5%), 239 indicated that they were female (49.4%), and 
34 (7%) did not respond to the request for gender identification. The distribution 
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of the genders varied considerably across the years in high school in the sample. 
Females outnumbered males in freshman and sophomore years, whereas males 
outnumbered females in junior and senior years. 

Ethnicity. A total of 184 (38%) respondents indicated that they were 
Hispanic or Latino/Latina, 58 (12%) indicated that they were White, 26 (5.4%) 
reported being Black or African American, 117 (24.2%) indicated that they were 
Asian, 7 (1.4%) reported that they were American Indian or Alaska Native, 13 
(2.7%) indicated that they were Native Hawaiian or Pacific Islander, and 79 
(16.3%) did not respond to the question. 

Descriptive Statistics 

The following tables and figures provide descriptive statistics for MESA 
participants on the following constructs: self-efficacy, perceptions, and interest. 
The descriptive statistics are presented in order to provide a snapshot of gender, 
grade, and ethnic differences for each of the prescribed constructs. 

Gender differences. Gender differences in the responses to the criterion 
subscales are consistent across the criterion subscales, with male respondents 
responding more positively than female respondents on self-efficacy, perception, 
and interest in engineering. This data is presented in Table 1. 

Table 1 


Gender Differences 



Male 

Female 

Self-efficacy 

3.89 

3.47 

Perception 

4.10 

3.92 

Interest 

3.68 

3.36 


Differences according to ethnicity. There were modest differences in 
mean responses to each of the criterion subscales among respondents from the 
respective ethnic groups. Means scores for each ethnic group are provided in 
Table 2. 
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Table 2 


Mean Subscale Scores by Ethnicity 



Hispanic or 
Latino/ 
Latina 
(n = 184) 

White 
{n = 58) 

Black or 
African 
American 
(n =26) 

Asian 
(n = 117) 

American 
Indian or 
Alaska Native 

(» = 7) 

Native 
Hawaiian or 
Pacific 
Islander 
(n = 13) 

Undes¬ 

ignated 

Self-efficacy 

3.57 

3.76 

3.83 

3.75 

3.34 

3.31 

3.73 

Perception 

3.97 

3.98 

4.05 

4.06 

3.95 

3.86 

3.93 

Interest 

3.48 

3.35 

3.67 

3.67 

3.26 

3.61 

3.64 


Mean scores on each of the three criterion subscales increased slightly among 
respondents in the respective ascending grade levels (see Table 3). It is 
important to keep in mind that the differences did not necessarily occur within a 
specific school. 

Table 3 


Mean Subscale Scores by Grade Level 



Freshman 

Sophomore 

J unior 

Senior 

Self-efficacy 

3.48 

3.68 

3.74 

3.79 

Perception 

3.84 

3.98 

4.03 

4.15 

Interest 

3.36 

3.55 

3.56 

3.59 


Outcomes assessment. As the work of the project evolved, it became 
increasingly obvious that a variety of other outcomes (not included in the self- 
efficacy, perceptions, and interest subscales) resulted from MESA participation. 
However, the identification of those outcomes was not as clearly specified in the 
literature, and these outcomes were not as readily measured as the three 
generally recognized constructs—self-efficacy, perception, and interest. Using 
analysis from focus-group interviews conducted, a fourth subscale was 
developed for the assessment of these outcomes, which included specific areas 
of cognitive development, growth in affective dimensions, career choices, plans 
for career preparation, and continuing personal development. The revised ESIPS 
instrument was pilot tested with 224 students deriving from the states of 
Washington, California, and Utah (Denson et al., 2014). The researchers utilized 
the techniques of principle component analysis in order to reduce the number of 
items into a “principle component" that would account for the most variance of 
the observed variables. The Kaiser-Meyer-Oklin (KMO) measure of sampling 
adequacy was calculated and found to be acceptable (0.951). Bartlett’s test of 
sphericity was significant %2 (741) = 6495, p < 0.001. The analysis produced 
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four components with eigenvalues greater than one. The Cronbach’s Alpha’s for 
the four components are: al = 0.928, a2 = 0.932, a3 = 0.897, a4 = 0.894 
(Denson et al., 2014). Analysis of the outcomes subscale provided several 
insights that offer opportunities for further investigations. 

Outcomes of MESA involvement. Responses to items on the outcomes 
subscale were analyzed by gender and ethnicity. The results of the gender 
analysis are presented in Table 4. Male respondents had a mean score of 3.89 on 
the outcomes items, and female respondents had a mean score of 3.78. 

Table 4 

Outcome Means by Gender _ 

Total Male Female 


Outcome means 3.82 3.89 3.78 


Respondents who identified as Flispanic of Latino/Latina and respondents 
who identified as Native Hawaiian or other Pacific Islander had the highest 
mean responses on the outcomes items, 3.88. Black or African American 
respondents and Asian respondents had mean responses of 3.82, White 
respondents had mean responses of 3.69, and American Indian or Alaska Native 
respondents had mean responses of 3.66 on the outcomes items. These results 

appear in Table 5. 


Table 5 

Outcomes Means by Ethnicity 


Ethnic Selection 

Mean 

Hispanic or Latino/Latina 

3.88 

White 

3.69 

Black or African American 

3.82 

Asian 

3.82 

American Indian or Alaska Native 

3.66 

Native Hawaiian or other Pacific Islander 

3.88 


Inferential Statistics 

Specific areas of MESA activities. Four categories of MESA activities 
were studied in more detail: hands-on activities, meeting professionals, student 
advisement, and field trips. Four facets of each of these activities were explored, 
each with a statement in this portion of the outcomes instrumentation. Using a 5- 
point Likert type scale, respondents were asked to indicate whether they strongly 
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disagree (1 point), disagree (2 points), neither disagree nor agree (3 points), 
agree (4 points), or strongly agree (5 points) with each statement. 

Correlations between participation in MESA activities and means on 
criteria subscales. One of the most fundamental questions addressed by this 
study was the influence of involvement in typical MESA activities and 
subsequent measures of self-efficacy, perceptions of engineering, and interest in 
engineering. That question is answered most directly by the significant 
correlations between the means of the hands-on activities, meeting 
professionals, student advisement, and field trips sections of the survey and the 
self-efficacy, perception, and interest subscales. Each category of activities was 
closely correlated with each of the criterion measures. These correlations are 
reported in Table 6. 

Table 6 


Patterns of Responses to Items Assessing Effects of MESA Activities 



Hands-on 

activities 

Meeting 

professionals 

Student 

advisement 

Field 

trips 

Self-efficacy 

0.46 

0.35 

0.26 

0.18 

Perceptions 

0.45 

0.36 

0.26 

0.28 

Interest 

0.51 

0.45 

0.39 

0.31 


Note. All probabilities < .05* 


The results of the correlation matrix illustrate a positive relationship between 
MESA’s activities and students’ self-efficacy, perceptions, and interest of 
engineering. Closer examination reveals that the most influential activity on the 
aforementioned constructs is hands-on activities. It should be noted that these 
comparisons help illustrate the difference in influence for aspects of the MESA 
program. Results from this analysis do not and should not be seen as predictors 
of success in other informal learning environments. 

Relationship between categories of MESA activities and criterion 
measures. Competitive events are among the optional opportunities available to 
MESA participants. The survey probed several aspects of the individuals’ 
involvement in MESA-sponsored competitive events. Respondents who 
indicated that they participated in competitive events had significantly higher 
mean scores on the self-efficacy, perception, and interest subscales of the 
survey. These results are included in Table 7. 
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Table 7 

Influence of Participation in MESA Competitions Upon Criterion Measures 



Yes 

(n = 309) 

No 

(n = 161) 

t 

p value 

Mean 

SD 

Mean 

SD 

Self-efficacy 

3.77 

0.72 

3.46 

0.77 

4.23 

p < .05* 

Perception 

4.07 

0.54 

3.88 

0.66 

3.15 

p < .05* 

Interest 

3.60 

0.75 

3.31 

0.76 

3.94 

p < .05* 


Note. Respondents’ participation was determined by their answer to the 
following question: “Did you participate in competitions at the club or class 


level?” 


Conclusion 

It was the researchers’ intent to provide some insight into the aspects of 
informal learning environments that may influence students and the type of 
events and activities that were particularly attractive to underrepresented student 
populations. Results of the study include the development of the ESIPS 
instrument, data from focus-group interviews, and quantitative research results. 
The mixed-method investigation of the MESA program yielded both 
quantitative and qualitative research findings and provided empirical evidence 
of MESA’s impact on underrepresented student populations. Results from focus- 
group interviews provided insight into the benefits of the MESA program for 
underrepresented students. These included: (a) informal mentoring, (b) makes 
learning fun, (c) time management, (d) application of math and science, (e) 
feelings of accomplishment, (f) builds confidence, (g) camaraderie, and (h) 
exposure to new opportunities (Denson, Hailey, Stallworth, & Householder, 
2015). The results of the qualitative research informed the development of the 
ESIPS instrument, which was used for the quantitative research phase (Denson 
et ah, 2014). 

The quantitative phase of this mixed-method study included administering 
the ESIPS instrument to over 700 student respondents. The study was able to use 
the results of 484 underrepresented students who had participated in MESA. The 
instrument measured students’ self-efficacy, interests, and perceptions related to 
engineering. Findings from this study appear to corroborate expectations that 
MESA activities and competitive events make substantial contributions toward 
the development of MESA participants. Inferential statistics provide evidence 
that participation in MESA provides positive influences on underrepresented 
students perceptions, interests, and self-efficacy related to engineering. This 
study examined the relationship between the aforementioned constructs and 
MESA’s hands-on activities, student advisement, meeting professionals, and 
field trips. The results revealed that each MESA activity was positively 
correlated with each of the constructs measured; however, hands-on activities 
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seemed to have the strongest impact on students’ self-efficacy, interest, and 
perceptions related to engineering. The study also revealed that students who 
were active participants in MESA had higher self-efficacy, interest, and 
perceptions related to engineering. 

The MESA respondents in this study appear to follow the general pattern of 
positive outcomes attributable to participation in structured extracurricular 
activities reported by Eccles, Barber, Stone, and Hunt (2003) and by Feldman 
and Matjasko (2005). The MESA experiences of the respondents were perceived 
as making contributions to their sense of self-efficacy in engineering, their 
perceptions of engineering, and their interests in engineering. Participation in 
MESA activities and MESA competitive events had positive outcomes in many 
dimensions explored in this study. Active involvement in these competitions and 
organized activities appears to contribute to the development of self-efficacy in 
engineering, to more accurate perceptions of engineering as a profession, and to 
enhancing interest in engineering as a field of study and as a career. 

The results of this study have several implications for informal learning 
environments and formal learning environments alike. The results of this study 
speak to the role of informal learning environments. Operating outside the 
constraints of standards-based testing and statewide curriculums, the MESA 
program seems to be effective in impacting students’ knowledge, skills, and 
affective abilities. Although the findings cannot be characterized as surprising, 
they do appear to corroborate expectations that MESA activities and competitive 
events make substantial contributions toward the development of MESA 
participants. Qualitative research results from this study provide empirical 
evidence as to mechanisms within the MESA organization that appear to make it 
effective. This is of particular beneficial for formal learning environments that 
struggle to introduce STEM-based content to underrepresented student 
populations. In addition, the ES1PS instrument developed for this study is fully 
adaptable to other engineering-focused informal learning environments and has 
proven to be a valid and reliable instrument when used in whole or in part. 
Finally, qualitative results from this study suggest that MESA’s activities have 
an influence on underrepresented students and that student’s benefit greatly from 
their hands-on experiences. Further research should focus on MESA’s ability to 
influence the teaching and learning of STEM content. 
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